\section{Virtualization}
Virtualization is an established architectural concept in computer science and
has been in use for decades. Operating systems for example provide virtual
address spaces to application processes, giving them the illusion of unlimited,
continuous memory. Hence an application does not need to take care of complex
memory management tasks and the operating system is able to optimize the usage
of physical memory. Another common example is the virtualization of devices such
as virtual CD-ROM drives directly using a file-based backend for data I/O
\cite{CryptoCloud}.

Hardware or platform virtualization is the process of simulating virtual
computer hardware that acts like real hardware. The virtualization is performed
and controlled by special software, called a
\emph{hypervisor}\index{hypervisor} or \emph{virtual machine monitor}
(VMM\index{VMM}). These two terms are synonyms; for consistency we will use VMM
throughout this document. VMMs are classified into two types, as shown in figure
\ref{fig:vmm-classification}.

\begin{figure}[h!]
	\centering
	\begin{subfigure}[b]{0.24\textwidth}
		\centering
		\input{graph_vmm_type1}
		\caption{\emph{Type I, native or bare metal VMM.} Runs directly on the
		hardware in the most privileged processor mode.}
	\end{subfigure}
	\qquad
	\begin{subfigure}[b]{0.24\textwidth}
		\centering
		\input{graph_vmm_type2}
		\caption{\emph{Type II or hosted VMM.} The VMM runs on top of a
		conventional operating system and uses OS services.}
	\end{subfigure}
	\caption{VMM classification}
	\label{fig:vmm-classification}
\end{figure}

The VMM runs on the \emph{host} and software that runs in a virtualized
environment is called \emph{guest} software. The terms host and guest are
generally used to distinguish the software that runs on the physical machine
from the software that runs on the virtual machine (VM\index{VM})
\cite{wiki:virtualization}.

By using virtualization and scheduling techniques, a VMM multiplexes the
hardware of a computer making it possible to run multiple guests
simultaneously\footnote{On multicore machines in parallel, on single-core
machines in pseudo-parallel} on a single physical machine. This is the most
common use-case for virtualization: the consolidation of operating system
instances on one server to save power and to improve hardware-resource
utilization.

A guest program is separated from the real hardware of a machine. Direct access
is restricted and can be controlled by the host VMM. Since the VMM has complete
control over the hardware and guest software state, virtualization is also
useful for separation purposes. The VMM is able to allow certain communication
channels between guest software while restricting others. One guest could have
access to the network card, while others share a page in memory but have no
access to hardware devices.

The principle idea of virtualization is to run guest code on a (virtual)
CPU\index{VCPU} and only intercept privileged operations accessing resources or
system properties for which direct access is prohibited. Various techniques
exist to intercept guest instructions, ranging from inspection and modification
of the guest software instruction stream (slow) to hardware-assisted
virtualization (fast). This thesis focuses on the hardware-assisted approach.
The reader is directed to \cite{VMware:virtualization} for further information
about virtualization mechanisms.

Independent of the chosen mechanism, interception of the running guest leads to
a so called \emph{trap}\index{trap} from the guest code into the VMM. The VMM
then examines the \emph{exit reason}\index{exit reason} and reacts accordingly
by for example modifying the guest software state and then resuming guest
execution.

This method is also used to implement a technique called "trap and emulate".
For example, if a guest accesses a device which is emulated by the VMM, a trap
into the VMM occurs. The VMM itself or a specialized VM monitor emulates the
requested operations of the guest software by directly modifying the state of
the virtual processor or memory of the trapping VM.

With hardware-assisted virtualization, traps are handled by the virtualization
hardware automatically. The exact behavior is configurable by the VMM software.

\subsection{Intel Virtualization Technology (VT)}
A trap from the guest into the VMM is a costly operation. To reduce traps,
modern processors have introduced mechanisms to support the VMM in creating a
virtual machine environment. These features not only improve the performance of
a virtual machine by avoiding traps, but also greatly simplify the VMM
implementation.

Intel Virtualization Technology (VT\index{VT}) provides hardware-assisted
virtualization mechanisms for Intel processors. Intel VT encompasses multiple
virtualization features:
\begin{itemize}
	\item Intel VT-x
	\item Intel EPT
	\item Intel VT-d
\end{itemize}

\subsubsection{Intel VT-x}
Intel VT-x\index{VT-x} provides a virtual machine architecture to allow
efficient processor virtualization. An Intel processor reports this feature
with the Virtual Machine Extensions (VMX\index{VMX}) CPU flag.

The virtual machine architecture is implemented by a new form of processor
operation called VMX operation. This mode is enabled by executing the
\texttt{VMXON}\index{VMXON} instruction and provides two operating modes: VMX
root operation and VMX non-root operation.

\begin{figure}[h]
	\centering
	\input{graph_vmm_lifecycle}
	\caption{Interaction of VMM and Guests}
	\label{fig:vmm-lifecycle}
\end{figure}

The VMX root operation mode is used by the VMM software while guest software
runs in VMX non-root operation mode. A transition from VMX root to VMX non-root
is called a VM entry, while a transition from VMX non-root to VMX root is called
a VM exit or trap. Figure \ref{fig:vmm-lifecycle} shows the life cycle of a VMM
and the transitions between guest software and the VMM.

Processor behavior in VMX root mode is very similar to non-VMX operation.  The
main difference is the extra instruction set provided by VMX, containing
virtualization-specific instructions used to manage VMX processor mode and
virtual machine control structures.

In VMX non-root operation however, the execution environment is restricted to
facilitate virtualization. Privileged or critical instructions cause VM exits
instead of their normal behavior. This allows the VMM to regain control of
processor resources and, depending on the trapping instruction, take
appropriate action such as emulating certain functionality.

The VMM software initiates a VM entry into guest code by executing the
\texttt{VMLAUNCH} and \texttt{VMRESUME} instructions. Only one guest can be
active on a logical processor at any given time. The VMM regains control using
the VM exit mechanism. If an exit occurs, the VMM analyzes the cause of the
exit and acts accordingly. It may resume the guest software or allocate
processor time to a different guest. The VMM exits VMX operation by calling the
\texttt{VMXOFF}\index{VMXOFF} instruction.

The Virtual-Machine Control Structure (VMCS\index{VMCS}) is used to control VMX
non-root operation and VMX transitions. Each virtual machine has an assigned
VMCS which allows fine-grained setup of processor behavior in VMX non-root
mode. A VMCS contains fields which can be written by the
\texttt{VMWRITE}\index{VMWRITE} instruction and read via
\texttt{VMREAD}\index{VMREAD} in VMX root mode. The fields can be categorized
into host-state, guest-state, read-only data and control fields.

On VM entry for example, the state of the host is saved automatically into the
subject VMCS by the VMX extensions and restored again on VM exit. On the other
hand, the guest-state area is used to save and restore guest state on VM exit
and VM entry respectively. The read-only data fields provide VMX status
information and the control fields in the VMCS govern VMX non-root operation.

For further information about the VMX processor extensions and VMCS, the reader
is directed to the respective section in the Intel SDM \cite{IntelSDM}, volume
3C, chapters 23 and 24.

\subsubsection{Intel EPT}\label{subsubsec:ept}
Extended Page Table (EPT\index{EPT}) is Intel's implementation of the Second
Level Address Translation (SLAT\index{SLAT}) virtualization technology. It
provides hardware-assisted translation of guest-physical memory addresses to
host-physical addresses. Guest-physical addresses are translated by traversing a
set of EPT paging structures provided by the VMM to produce physical addresses
that are used to access memory.

The EPT paging structure confines the host-physical memory region that a guest
virtual machine is allowed to access, thereby making it possible to safely run
unmodified guest OS memory management code. Access to host-physical memory
outside of the allowed region results in an EPT violation trap.

Without EPT, the burden of translating guest-physical addresses to
host-physical addresses while guaranteeing guest/vmm and guest/guest memory
space separation rests with the VMM. This is a non-trivial task which is highly
inefficient. EPT removes the complexity of manual memory address translations
via shadow page tables from the VMM, making the code simpler while improving
virtualization speed. For more information about EPT see Intel SDM
\cite{IntelSDM}, volume 3C, section 28.2.

\subsubsection{Intel VT-d}\label{subsubsec:vtd}
Virtualization Technology for Directed I/O (VT-d\index{VT-d}) provides hardware
support for I/O-device virtualization. While VT-x provides the support to
virtualize the platform (i.e. the processor), VT-d is used to simplify the
direct assignment of devices to virtual machines by providing direct memory
access (DMA\index{DMA}) and device interrupt remapping functionality. VT-d
provides an I/O memory management unit (IOMMU\index{IOMMU}) required to control
device DMA.

The IOMMU implements address translation functionality similar to the MMU, but
for accesses to system memory initiated by I/O devices.  This is necessary
since PCI devices can perform DMA which bypasses the processor's MMU memory
protection. Like the MMU, an operating system must initialize data structures
and set up the IOMMU for proper separation of physical memory accessible by
devices.

The VT-d technology is outside the scope of this master thesis, the reader is
directed to \cite{IntelVTd} for a more in-depth discussion.
